Dba 738 index optimization by llm #64
Conversation
| response = client.chat.completions.create( | ||
| model="gpt-4o", | ||
| response_model=IndexingAlternative, | ||
| temperature=1.2, |
There was a problem hiding this comment.
are you sure you want above average "temperature"? May make it too "creative"
There was a problem hiding this comment.
There is no way to know what the right value is without testing. My inclination is to err on the creative side so that we explore the space of indexing recommendations. I have occasionally seen recommendations that refer to tables that don't exist or are otherwise invalid. They get rejected and my guess is that having some of that is healthy.
| else: | ||
| remaining_attempts_prompt = "" | ||
|
|
||
| response = client.chat.completions.create( |
There was a problem hiding this comment.
Consider litellm to not hardcode the model https://docs.litellm.ai/#basic-usage . They seem to map various apis to the open-ai completion format so rest of code shouldn't need to change, and it will pull api key from standard env variables (per provider)
There was a problem hiding this comment.
We still need to put in code to switch the the desired model, right?
I.e., somewhere we need to see what is available, say via env variables, and then use that model. I don't think LiteLLM will do that for us.
3 participants
This is a draft implementation of optimization by LLM.